Noise is one of the main causes of degradation in images (e.g., video and still images) captured by image sensors. Conventional noise filtering techniques typically apply various averaging or smoothing operations to suppress noise, under the assumption that noise is random and unstructured such that it can be canceled out by averaging or smoothing.
However, the assumption of unstructured randomness of noise is not accurate. In fact, noise may include both a fixed pattern noise (FPN) component (e.g., due to column noise in readout circuitry, irregular pixel sizes, and/or other irregularities) and a random noise component. The FPN component may appear as a noisy pattern that is essentially constant through time, and as such it is not attenuated by averaging, but often becomes even more visible after conventional noise filtering. The FPN becomes more problematic for low-cost sensors, sensors with extremely narrow pixel-pitch, or sensors operating in implementations with a very low signal-to-noise ratios (SNRs) (e.g., in low-light imaging, thermal imaging, range imaging, or other imaging applications with low SNRs). Furthermore, for most imagers, both the FPN and random noise components are typically structured (e.g., colored noise), with different correlations present in the FPN and random noise components. Thus, conventional filtering techniques often produce images with prominent structured artifacts.
In addition to random and fixed pattern noise, images may contain distortion and degradation caused by atmospheric turbulence as light travels through the air from the source to an image sensor, which may particularly be noticeable in outdoor and/or long distance images. For example, variations in the refractive index in turbulent air may cause image blur randomly varying in space and time, large-magnitude shifts (also referred to as “dancing”) of image patches also randomly varying in space and time, and random geometrical distortion (also referred to as “random warping”) of images.
Conventional techniques such as bispectrum imaging, lucky imaging, and temporal averaging have been developed to address at least some distortion caused by atmospheric turbulence. However, such conventional techniques require static scenes (e.g., a plurality of short-exposure frames of the same static scene) to work. While such conventional techniques may be adapted to work on scenes with motion or moving objects by applying the techniques on a sliding temporal window basis, this produces various undesirable results. For example, applying such conventional techniques to scenes with motion or moving objects typically leads to motion blur as well as ghosting effects. Further in this regard, high temporal frequency content is lost from scenes with motion when such conventional techniques are applied, in addition to losing high spatial frequency content.